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Abstract — Random coding, along with various standard tech- 
niques such as coded time-sharing, rate-splitting, superposition 
coding, and binning, are traditionally used in obtaining achiev- 
able rate regions for multi-terminal networks. The error analysis 
of such an achievable scheme relies heavily on the properties of 
strongly joint typical sequences and on bounds of the cardinality 
of typical sets. In this work, we obtain an achievable rate 
region for a general (i.e. an arbitrary set of messages shared 
amongst encoding nodes, which transmit to arbitrary decoding 
nodes) memoryless, single-hop, multi-terminal network without 
feedback or cooperation by introducing a general framework 
and notation, and carefully generalizing the derivation of the 
error analysis. We show that this general inner bound may be 
obtained from a graph representation that captures the statistical 
relationship among codewords and allows one to readily obtain 
the rate bounds that define the achievable rate region. The 
proposed graph representation naturally leads to the derivation 
of all the achievable schemes that can be generated by combining 
classic random coding techniques for any memoryless network 
used without feedback or cooperation. 

Index Terms — achievable region, chain graph, multi-terminal 
network, superposition, binning, rate-splitting, time-sharing. 

I. Introduction 

In random coding, codewords are generated by drawing 
symbols in an independent, identically distributed (iid) fashion 
from a prescribed distribution; the performance of the ensem- 
ble of codes is then analyzed as a function of the block- 
length, which is eventually taken to infinity. Thanks to the 
iid symbols, and a block-length which tends to infinity, it is 
possible to derive the asymptotic performance of the ensemble 
of codes using the properties of jointly typical sets [Q. This 
proving technique was originally developed for the point-to- 
point channel |2] but is easily extended to multi-user channels 
by introducing multiple codebooks, one for each message to 
be transmitted. Multiple codebooks can be further organized 
in complex schemes that takes into account the structure of 
the network and the distribution of the messages among the 
users. Coded time-sharing, rate- spitting, superposition coding, 
binning, Markov encoding, quantize and forward, are some 
of the strategies that have been developed for multi-terminal 
channels. Given that all achievability schemes tend to use a 
combination of "standard" techniques applied in different fash- 
ions (leading to different dependencies amongst codewords), 
one might expect to be able to derive a general achievability 
scheme for a large class of networks. The key bounding tech- 



niques to analyze the error probability of transmission schemes 
are presented in an unified fashion by Csiszar and Korner d 
Ch. 1.2] and, more recently, by Kramer (51 Ch. 1] and El 
Gamal [4, Ch. 1]. In this paper we introduce an achievable 
scheme involving superposition coding, binning, rate- splitting, 
and coded time- sharing valid for a general one-hop channel 
without feedback or cooperation. This achievable scheme is 
defined by the random variables representing different code- 
words and by the factorization of the joint distribution among 
these random variables. We develop a graph representation 
of the factorization of the joint distribution based on Markov 
graphs 0, (6): in particular we define a Markov chain graph 
with two types of edges, one representing superposition coding 
and one representing binning. 

In information theory, a similar attempt to capture the 
relationship between random variables is the functional depen- 
dence graph |3, App. 8]. Despite fundamental similarities, the 
functional dependence graph is used mainly to represent the 
Markov relationship between random variables, while, in our 
case, the Markov chain graph describes a transmission scheme 
and the relationship between codewords of different users. 
Another attempt to provide achievable regions for general 
channels is hinted in (7), but the authors do not provide a 
rigorous approach to the problem. 

By building upon the fundamental results in random coding 
theory and graph theory, we define a formal representation and 
a standard notation for a general achievable scheme as well 
as the derivation of the achievable region. Our ultimate goal 
is to define a form of "automatic rate region generator" which 
outputs the best known random coding achievable rate region 
for any channel of choice. 

Paper Organization 

Section [III presents the class of networks considered in 
this work and revises the standard random coding techniques 
that we employ in our general achievable scheme. Section Hill 
introduces the novel chain graph representation of the encod- 
ing operations. Section HVl describes the codebook generation, 
encoding and decoding procedures of the achievable scheme 
associated to a specific graph representation. Section W\ derives 
the rate bounds that define the achievable rate region based on 
the proposed chain graph representation. Section [VTl concludes 
the paper. 
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II. Channel Model and Random Coding 
Techniques for Achievability 

A. Notation 

In the following, bold letters indicate subsets of power sets 
% i.e. 



icy N = y([i...N])> 



(i) 



• index k/z: transmitter/receiver index with Xk,Y z being the 
channel input/output at transmitter k and receiver z, 

• index i/j: subset of transmitters/receivers. We also use 1/m 
and v/t, 

• index S: set of containing (i, j) pairs and S its complement. 

B. Network Model 

We consider a general multi-terminal network where TVtx 
transmitting nodes want to communicate with TVrx receiving 
nodes. A given node may only be a transmitting or a receiving 
node , that is, the network is single-hop and without feedback. 
The transmitting node fc, k G [1 . . . iVrxL inputs X^ to the 
channel, while the receiving node z, z G [1...7Vrx], has 
access to the channel output Y z . The channel transition prob- 
ability is indicated with P Yl ...y n I x ± . . .x N an d the channel 
is assumed to be memoryless. The subset of transmitting nodes 
i, i G ^Pat tx , is interested in sending the message W\^j to the 
subset of receiving nodes j G *Pat rx over N channel uses. The 
message W\^, (i,j) G *Pat tx x ^Ptv rx > uniformly distributed 
in the interval [0 . . . 2^^^ — 1], where N is the block-length 
and Ri^j the transmission rate. 

A rate vector R = {Ri^j, V (i,j) G ^Pat tx x *Pat rx } is 
said to be achievable if there exists a sequence of encoding 
functions 

X» = X», ({WUj, s.t. (i, j) c y NTX x y NRX , k e i}) , 

and a sequence of decoding functions 



\ z . = W> z • (} 



WL; 



-N 



Y?)i£zel 



(2) 



such that 



lim max I 

iV— )-oo i,j,z 



W- z • 4 W- z • 



0. 



The capacity region 6(R) is the convex closure of the region 
of all achievable rates in the vector R-pairs. The general 
network model we consider is a variation to the network model 
in (U Ch. 14], but we allow for messages to be distributed to 
more than one user while not considering feedback. 

Fig. [T] shows the channel model considered in this work. 

C. Random Coding Techniques for Achievability 

We now revise standard random coding techniques used 
in the literature for achievability in single-hop networks 
used without feedback/cooperation: coded- time- sharing, rate- 
splitting, superposition coding, and binning. 
• Coded Time-Sharing consists of using different transmis- 
sion strategies according to a random schedule |9] and allows 
one to achieve the convex closure of the set of achievable 
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Fig. 1. The general cognitive multi-terminal network. 

points and, in some cases, an even larger region [1, pp. 288- 
290] . Let q N denote denote the outcome of N iid draws from 
the distribution Pq and let q N be revealed to all the encoders 
and decoders in the network. In coded time- sharing the users 
generate multiple random codebooks conditioned on the value 
of the sequence q n and choose their transmission strategy 
according to this random outcome. 

• Rate-Splitting corresponds to dividing a message Wi^j into 
a set of sub-messages {W^j/} where j C j' and i D i', 
that is, sub-messages that are encoded by a smaller subset 
of transmitters and decoded at a larger set of receivers than 
the original message. Rate splitting preserves the rate of the 
transmitted messages but increases the number of messages to 
be transmitted over the channel which, in terms, increases the 
possible transmission strategies (9). Let W/^ indicate that 
sub-message Wi^ m obtained by splitting the message W\^, 
then the message Wi^j can be split in a sequence of sub- 
messages Wy 1 ^ for every (1, m) such that jcm and i D 1 
so that 



R[ 



'i-M 
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(l,m) jCm, iDl. 
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i), jCm, iDl. 
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Ri^h 



(3) 



with 7) _ J = *^ m and where R(. is the rate of the RV 
Wi^j in the original channel and R^j in the rate of the RV 
W[^j in the channel after rate-splitting is applied. 

Rate- splitting effectively transforms the problem of achiev- 
ing a rate vector R into the problem of achieving the rate 
vector R' such that R 7 = TR with r(ij) x ( 1?m ) = 7)!^. 
• Superposition Coding can be intuitively thought of as 
stacking codewords on top of each other ifTOl . The "base" 
codewords are decoded first and stripped from the received 
signal so as to reduce the interference when decoding the "top" 
codewords. In superposition coding a different top codebook 
is generated for each base codeword and the codewords in 
the top codebook have a conditional distribution that depends 
on the specific base codeword. Let U^ m be the codebook 
with distribution Pu l ^ m carrying the message W/_> v and the 
codeword U^ carrying the message W[^ be superposed to 
. A different top codebook is associated 
and the codewords 



the codebook U^ 

to each base codeword U^ 



,N 



C/j^j in each of these codebooks are generated according to 
the distribution PJf ]TT oi . Superposition of U^: over 
^i^m can ^ e performed when the following hold: 

• 1 C i: that is the bottom message is encoded by a larger set 
of encoders than the top message. 

• mCj: that is the bottom message is decoded by a larger 
set of decoders than the top message. 



N 



T N 



Note that, if U(^- is superposed to U{^ m and U^ t is 



superposed to U{^ m , than U^- is also superposed to U^ t . In 
the following [/i_>j <U\^ m indicates that the codeword U^- is 
superposed to the codeword U{^ m . 

• Binning allows a transmitter to "pre-cancel" (portions of) 
the interference known to be experienced at a receiver lITTIl . 
This is done by generating a codebook that has a larger number 
of codewords than the cardinality of the associated message 
so that the transmitted codeword can be chosen according to 
both the message and the specific value of the interference. 
Although in the original setting of ifTTTl the interference is a 
random process, this technique can be extended to the case 
where the interference is a different user's codewords. Let 
U{^ m be the codeword with distribution Pjj l ^ m carrying the 
message W/_, v and the codeword C/j^. carrying the message 



Wi 



tN 



N 



' • be binned against U{^ m . The codewords U{ y - are gener- 



ated independently from the codewords U{^ m but chosen so to 
look as if generated according to the distribution P^ , v 
The codeword U-^- can be binned against U{^ m when 

• i C 1: that is, the set of encoders performing binning has 
knowledge of the interfering codeword 

Note that if U^- can be binned against U^ m , then U^ can 
also be binned against U-^ m , : this is referred to as "joint 
binning" [12]. In the following Ui^j < U\^ m indicates that the 
codeword U^- is binned against the codeword U{^ m while 
Ui^j < >U\^ m indicates joint binning. 

III. The Chain Graph Representation of a General 
Achievable Schemes 

The elements included in the random coding construction of 
Sec. III-CI may be compactly represented using the following 
graph 9(V,E): 

• every vertex v = (i,j) € V C ^P(iV T x) x *P(^rx) is 
associated to the RV U-^j carrying the message W( • at rate 
R[' obtained through rate- splitting, 

• the set of edges E, connecting the vertices in V, contains 
two subsets S and B such that E = S{JB<mdSPiB = ® 

• the vertex Ui^j is connected with U\^ m by a directed edge 
of type S (for superposition), if U\^ m <Ui^j (solid line). 

• the vertex U^ is connected with C/ v ->t by a directed edge 
of type B (for binning), if U v ^ t < EA->j (dotted line). 
Moreover, all the RVs are generated according to a marginal 
distribution that depends on Q. 

A schematic representation of the graphical Markov model 
associated with the graph S(V, E) is presented in Fig. [2 
each vertex (i, j) is associated with the message W/_>j, a rate- 
splitting equation and an auxiliary RV U\^ with distribution 
P[/. jq according to which the codebook is generated. The 
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Fig. 2. A schematic representation of the graph in Sec. [nil . 



^i->j - £j(l,m), (7) Tl^m^HJ 



vertex (i, j) can be connected to a vertex (1, m) by two types 
of edges: a superposition edge S and a binning edge B\ both 
edges indicate the Markov dependency between U\^ and 
f/i-m given Q. 

The graph representation of the achievable scheme is partic- 
ularly useful in deriving the joint distribution of the codewords 
C/j^j in a general scheme in Sec. HH When trying to determine 
this distribution, superposition coding and binning effectively 
result in allowing for any joint distribution among the con- 
nected RVs. In the following we detail how the graph S(V, E) 
can be used to describe the dependency structure of the code- 
words of an achievable scheme for a general multi-terminal 
network. Graphs representing conditional dependencies among 
RVs have been extensively studied in the literature in the 
field of graphical Markov models |5|. In order for the graph 
representation of achievable scheme to correspond to a feasible 
distribution, it is necessary to impose certain restrictions on 
its structure. Moreover, it is convenient to consider graph 
representations where there exists a convenient factorization 
of the joint distribution of the RVs in the graph. For this 
reason we consider a decomposable chain graph: graph with 
both directed and undirected edges that is equivalent to some 
Asymmetric Directed Graphs (ADGs). 

Assumption 1. RVs that are jointly binned form fully con- 
nected sets, that is 

L7i-j < >£A^ m , U-^m < >E/i-j => U-^ m < >EW (4) 

Assumption 2. RVs that are jointly binned, have the same 
parent nodes, that is, if L^j < >Ui^t and Ui^j < U\^ m or 
Ui^j<U\^ m , than U-^t < U\^ m or Ui^ t <U\^ m . 

Assumption 3. RVs known at same set of decoders i do not 
form directed cycles in the graph representation. If a cycle 
exists it must be undirected. 

Given any achievable scheme, one can always obtain an 
achievable scheme that satisfies Assumptions [TJ [2] and [3] by 
adding an additional binning steps to the original scheme. 
These additional encoding operations can only enlarge the 
achievable region, since the original scheme can be reobtained 
with the appropriate choice of distribution imposed by the 
binning step. 



Theorem III.l. If Assumptions \7\ \3\ and\2\ hold, the graph 
representation of a general achievable scheme corresponds to 
chain graph which is equivalent to some ADG for which the 
joint distribution can be written as 



p(v) = n Pi 



£/ fc |pa(U k )> 



(5) 



u k ev 



where pa(Uk) indicates the parent nodes ofUk, that is the set 
of all the nodes connected to Uk by a direct edge. 

Proof: The complete proof is provided in |[T3ll . ■ 

We refer to a graph representation of an achievable scheme 
in Th. IIII. II as a Chain Graph Representation of an Achievable 
Scheme (CGRAS). 

The edges of the equivalent ADG are generated by orienting 
undirected edge and therefore the equivalent ADG can be 
written as 3(V,B- U S) for some B- C B. The notation < 
is used in the following to indicate the edges in B-. With 
this notation, the joint distribution of a CGRAS can then be 
factorized as: 



{u^, v (i,j)} 



(U) 



U^<U^ m , [/i.j¥[/i. m }' 



(6) 



IV. Codebook Generation, Encoding and Decoding 

PROCEDURES FOR A CGRAS 

We now outline the codebook generation, encoding, and 
decoding operations for a general transmission scheme asso- 
ciated with a CGRAS. 

Codebook Generation: The codebook of a CGRAS is 
generated according to the distribution imposed by the super- 
position coding edges. Consider the node L^j and assume that 
the codebook of the parent nodes has already been generated 
and indexed by /i^ m G [1 . . . 2 Ll ^ m ], then, for each possible 
set 

{Zi^n, V(l,m)E^j<£^ m }, 

repeat the following: 

• generate 2 NLi ^ codewords, for Li^j = R[ . + 
i?i_Kj with iid symbols drawn from the distribution 
P U^\{U^ m , u^ m <u^ s }- ^ _ f 

• Place the codewords in 2^^-^ bins of size 2 NR{ ^ each. 

• Index each codebook of size 2 NLi ^ using the set 
{ Zi_> m , V (l,m) E/i-j<Ui- m } so that 



the number of bins fri^j is sufficiently large, that is, if R\^j is 
sufficiently large. Finally the encoder k produces the channel 
input Xj^ as a deterministic function of its codebook(s). 
Decoding Procedure Receiver z looks for a set of bin 



indices w\_ 



for z G j, such that the set 



. . and b ^J 

I Y^ , \ C/j^j : z E j|| looks as if it generated an iid ac- 
cording to the distribution in ©. If the decoder cannot find 
such codeword or it finds more than one, it picks one tuple at 
random. 

V. Derivation of the Rate Bounds 

In this section we derive the achievable rate region of the 
transmission strategy described in Sec. |IV] by bounding the 
encoding and decoding error probability. 

A. Encoding Errors 

For the probability of encoding error to vanish as the block- 
length increases it is necessary to choose a large enough 
binning rate R\^ so the encoders can jointly find a set of 
bin index bi^j for which the codeword U^- appears to be 
generated according to the distribution imposed by binning 
although it is generated according to the distribution imposed 
by superposition coding. Define 



Sb = {(i, j), E/i»j < E/i-m for some (1, m)} , 



(8) 



^j(^j) = tf£j KLj,^j,{ h-* m , V (l,m) C/Uj<E/U»}) 

(7) 

where w[. is chosen according to the transmitted message. 
Since graph S(V,S) is an ADG, we can apply the above step 
starting from the nodes that have no parents to all the nodes 
in the graph. 

Encoding Procedure: The binning index bi^j in each 
codeword is chosen so that the codewords in d7]) appear to have 
been generated with iid symbols drawn from the distribution in 
©. If a codeword cannot be determined, then a value of b\^j 
is chosen randomly. We may find a jointly typical codeword if 



i.e. Sb is the set of all the indexes whose codeword possess a 
bin index; we intuitively expect the condition for a successful 
encoding to depend on the distance between the codewords' 
distribution at generation and after encoding: 

CdS = £pOg Encoding] - E[l0g P codeb ook] = (9) 

J2 i(0Uj;{tfi-m, Ui^U^m}\{U^ m , U^<U^ m }\Q). 

(U)es B 

The encoding error analysis is obtained using Markov 
inequality and the mutual covering lemma l3lh |4|. 

Theorem V.l. Encoding Errors Analysis using the Mutual 
Covering Lemma For any CGRAS, encoding is successful 
with high probability as N — >• oo if, for any subset S C Sb 
such that 

(i,j) e S =* (l,m) 6 S, V (l,m) s.t. U^ m <U^ h (10) 



for Sb defined in ^, the following holds: 



(ij) € S 



R^.\ > r 



encoding 
codebook 



i-M.C.L. 



(ID 



where I^-C.l. ^ defined in (fT2l) ( M.C.L. standing for ^Mutual 
Covering Lemma"). 

Proof: The complete proof is provided in |[T3l . ■ 



jM.c.l. = J- /([/ H ;{^ m , [/ H Wi. m ,(l,m)eS}|{[/,. m , [/ H <f/,. m , [/ hm <f/ H) (l,m)eS},Q), (12) 
(ij) e s 



r z,P.L. 



I(Y z ;{Ui^,(i,j) e S}|{C/,^ m ,(l,m) e S},Q) 

+ J2 %j;{^m.^^» 1 (i,m)es}|{^ m , u^<u^ m e s},g), 

(ij) e s 



(13) 



5. Decoding Errors 

For the decoding error probability to vanish as the block- 
length increases, it is necessary to choose a small enough 
codebook rate L\^j so that the codewords U^ are sufficiently 
"spaced apart" in the typical set to allow successful decoding. 
The bounds on the codebook rates are obtained using the 
packing lemma 0, (4) which bounds the maximum number of 
codewords that can be employed at transmission and still allow 
the decoder to recover the transmitted codeword. Decoding 
relies on the conditional typicality of the channel output given 
the transmitted codeword and on the typicality relationship 
imposed on the codewords by the encoding procedure. The 
probability of error at each decoder is linked to the specific 
codewords that the receiver is attempting to decode; for this 
reason it is convenient to define the set Sf) as: 



Sf) = {(i,j) s.t. zGJ}, 



(14) 



that is, Sf) is the set of indexes whose codeword is decoded at 
receiver z. Intuitively we expect the condition for successful 
decoding to depend on the distance between the joint distribu- 
tion of the codewords and the channel output at the encoding 
and the distribution after an incorrect decoding, that is: 

lading = £[logPy„encoding*] " E [log P Yz P£ debook ] 

= I(y 2 ;{C/Uj, (U) e Si,}|Q)+ 
J2 I(U^y,{U^ m , L/Uj^C>U m }|{E/U m , E>Uj<tfU. 

(ij)esf, 

where 

Py z .encoding* = Pr^U^eS^} ,Q P£ncoding ( 15a ) 

(15b) 
(15c) 



pz 

encoding 



/ -* encoding 



^codebook — / j -^codebook? 
(U)#3f> 

that is, P e 2 ncoding and P* odebook are the encoding and the 
codebook distribution, for the set of codewords decoded at 
receiver z, {E/i_>j, (i, j) e S^}. 

Theorem V.2. Decoding error analysis using the Packing 
Lemma 

For any CGRAS, decoding is successful with high proba- 
bility as N —> oo if , for any receiver z and for any subset 
S C Sf) such that condition (ITOb , for Sf) defined in SlAi , the 
following holds: 



^ ii.j < if D 



(U)es 



s > 



(16) 



where Ig " is defined in ([T3l) . (where P.L stands for 
packing lemma). 



Proof: The complete proof is provided in |[T3ll . ■ 

VI. Conclusion 

In this paper we present a new general achievable rate 
region valid for a general class of multi- terminal networks. 
This achievable scheme employs rate- splitting, superposition 
coding, and binning, and generalizes a number of inner bounds 
and techniques that have been proposed in the literature. 
This achievable scheme may be represented using a graph 
representation that allows for a quick comparison between 
transmission strategies and simplifies the derivation of the 
corresponding achievable rate regions. This paper attempts to 
establish a general tool to derive achievable rate regions for 
multi-terminal networks which contains all standard random 
coding techniques. A subject of ongoing research is whether 
there exists a combination of encoding strategies that yields 
the largest achievable region among all possible transmission 
strategies (within the proposed framework). 
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